Sign languages are natural languages mainly used by hard-of-hearing (HoH) people. While spoken languages use voices to convey meaning, sign languages utilize hand shape, orientation and movement, sometimes with the aid of facial expression and body movement, to express thoughts. Modern research has proved that sign languages are true languages and can be the first languages of HoH people.
The World Health Organization estimates that over 5% of the global population (including 32 million children) has disabling hearing loss. The technology of SLR may lead to products that can benefit the HoH communities in various ways. However, compared with SR, SLR has received relatively limited research efforts and is immature yet. Introducing the mature SR techniques into SLR may lead to substantial progresses in SLR.
The tasks of SLR and SR are similar in the sense that they both attempt to transfer a sequence of input signals into a sequence of text words. While the input signals for SR are voice signals, those for SLR could be sensor signals from wearable device and/or image sequences from a camera. The similarity indicates that the SR frameworks can also be applicable for SLR. Though, certain strategies need to be used to handle special characteristics of sign languages.
One major characteristic of sign language is that in the signals of a sign phrases, there are transition parts of signals between every two sequential signs, before the first sign and after the last sign. In previous works for small-vocabulary continuous SLR, the transition parts are normally ignored in modeling. However for larger-vocabulary continuous SLR, modeling transition parts explicitly has been proven to be beneficial. The previous works that explicitly model transition signals require transition models structurally different from sign models and significantly increase the complexity of recognition. Consequently, improvements to SLR systems and methods that improve the recognition of sign language gestures for sign languages with a wide range of vocabulary sizes would be beneficial.
For sign languages, there are other special characteristics related to the non-dominant hand. The first characteristic is that the dominant hand and the non-dominant hand are not equally important for sign languages. For example, frequently used sign-language words normally only involve the dominant hand in signing. The second one is that for the dominant-hand-only signs, the non-dominant-hand signals may not be stable and are influenced by the neighborhood signs or even additional actions. Strategies to handle these characteristics related to the non-dominant hand are thus beneficial for sign language recognition.